REMARKS 

Introduction 

In the Office Action of December 16, 2004, Claims 1-32 were rejected under 35 U.S.C. 
§ 102(b) as being unpatentable over U.S. Patent No. 6,029,195 to Herz. Pursuant to 37 C.F.R. 
§ 1.111, and for the reasons set forth below, applicant respectfully requests reconsideration and 
allowance of this application. Prior to discussing the reasons why applicant believes that the 
claims now pending in this application are allowable, a brief summary of the present invention is 
provided. The summary is not intended to define the scope or interpretation of any of the claims 
of this application. Instead, the summary is provided to help the U.S. Patent and Trademark 
Office better appreciate important claim distinctions between the present invention and the cited 
references. 

Sunmiarv of the Invention 

The present invention is directed to a system for generating recommendations. In 
accordance with one aspect of the invention, a genetic algorithm is utilized to adjust sets of 
parameters that are used for providing the recommendations. As described with respect to 
FIGURE 7, beginning on page 15, line 14 of the specification, in one embodiment the blocks 750 
and 760 of FIGURE 7 are implemented through a Genetic algorithm that works by evolving the 
fitness of a population of parameter sets towards ever higher fitness. The population is described 
as a group of parameters sets used by the system in question. Each parameter set represents one 
possible solution to the problem at hand. Over time, the more fit solutions are given more chances 
to breed than are the less fit solutions. The breeding process combines elements of the effective 
solutions to form new parameter sets. In one embodiment, (which may or may not be 
implemented in block 760), mutations in the parameter sets allow novel solutions to enter the 
population, thus allowing the consideration of solutions that were not initially conceived of prior 
to the training of the system. 
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In order for the population to evolve in a useful direction, the system that is dependent on 
the population must define what it means for an individual parameter set (hereafter referred to as 
an individual) to be fit. The fitness value of an individual may not be so important. Rather, the 
relative fitness values between individuals in a population may be more important. For the 
recommendation engine, in one embodiment the fitness of a given individual is determined by the 
percentage of recommendations rated as positive from a given individual parameter set. The user 
rating process provides the necessary feedback to the genetic algorithm to determine fitness. 

The first task to tackle when using the genetic algorithm is to specify the format of 
solutions. Each solution is typically represented by a genome. A genome is a collection of genes. 
Each gene represents some piece of information about the solution. Li one embodiment, the only 
type of gene supported is the floating point gene. This gene is a convenient format for solutions 
that are expressed as a set of floating point numbers. In another embodiment, binary genes may be 
implemented. 

In one example implementation, once the solution format, the genome structure, is 
established, the initial seed population may be created. In this particular example, this seed 
population may be provided by the user. (In another embodiment, a utility to generate initial 
populations is used.) The seed population represents the user's a priori information about the 
problem space. If the user is completely ignorant about the solution space, then an initially 
random seed population can be used. 

The size of the population will depend on the complexity of the problem being solved. 
The richer the parameter set, the more individuals the user will need in the initial population to 
sample a reasonable segment of the parameter space. In one embodiment, the recommendation 
engine maintains a population of about 50 individuals (at times there might be a few less than 
50 individuals, however, in this particular example there are generally not more than 
50 individuals). 

LAW OFFICES OF 
CHRISTENSEN O'CONNOR JOHNSON KINDNESS"^ 
1420 Fifth Avenue 
Suite 2800 
Seattle, Washington 98101 
.g. 206.682.8100 

MSFn20706AM.OOC 



Once the seed population is in place, the population is evolved through a series of epochs. 
Each epoch contains five key stages: fitness evaluation, death, breeding, mutation, and work. 
These stages are described in detail below. 

The fitness of every individual in the seed population is evaluated using a specified fitness 
fimction. The purpose of the fitness function is to form a basis for two decisions made by the 
genetic algorithm: Will the individual breed? Will the individual survive? The fitness fiinction is 
the genetic algorithm's main connection to the problem being solved. The fitness fiinction used in 
the recommendation engine is described in the next section. 

After the fitness evaluation, it is time for a death (a.k.a. survival) fiinction to prune the 
unsuitable individuals from the population. The probability of an individual surviving from one 
epoch to the next is given by 



where 5 is the survival rate and f is the fitness of the respective individual. 

For those who survive the death fimction, the breeding session begins. The probabihty of 
two individuals breeding is determined by two factors. The first is the fitness values of the two 
individuals. The fitness value represents the environmental bias the system has for an individual 
finding a mate. For example, in an organic example, the food supply, population densities, health 
of the individual, etc., all affect the ability of an individual to find a suitable mate. The probability 
that individuals A and B find each other and mate is given by the joint probability 





where a is the breeding rate and / is the fitness of the respective individual. 
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The second factor is the individual bias an individual holds for a particular mate. In a 
human example, such factors as appearance, wealth, personahty, etc., might affect an individual's 
bias towards an individual. In the genetic algorithm, the personal bias can be used for more 
practical tasks. For instance, in one embodiment individuals could be modified to favor those 
individuals that are similar to them, but not too similar, when mating. Such behavior is observed 
in many animal populations and acts to concentrate traits in particular populations. In the genetic 
algorithm, this effect could be exploited to encourage population clustering. In one embodiment, 
the personal bias, while it can be implemented in a Genetic Algorithm, is ignored by an individual 
object. Instead, in such an embodiment the individual would assume that if a proposed mate is fit 
enough to find him/her/it, he/she/it is good enough. 

Once the mating partners are selected, their genes are combined to form a new individual 
parameter set. The gene combining algorithm, or splicing, is performed by selecting a random 
point along the genome of an individual and replacing all genes after that point with the genes of 
the individual's mate. 

In one embodiment, a newly minted genome has some probability of undergoing some 
form of mutation. The probability that a gene is mutated as well as the standard deviation of the 
mutation probabihty distribution fiinction is specified by the user. Mutations allow a population to 
explore new segments of a solution space. Without mutations, a population's characteristics would 
eventually fi-eeze. However, mutations can destroy quality solutions. Therefore, the mutation rate 
and depth should be chosen carefiiUy. In one example, typical mutation rates are between 1 and 
10 mutations per 1000 genes evaluated. The mutation depth for a binary gene is in some cases 
considered to be irrelevant; if the gene mutates, it becomes its complement. For floating point 
genes, in one example the mutation depth could be on the order of a few percent. In other words, 
the value of the floating point number changes by a few percent. 
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Finally, the population is available for work. At this point the system retrieves the 
population and uses the information encoded in the genomes of the individuals to accomplish 
some task. In the case of the recommendation engine, the task is generating recommendations for 
various media items, such as illustrated at block 720 of FIGURE 7, Following the work stage, a 
new epoch is entered and the process is repeated. Thus, the routine 700, and the use of the Genetic 
Algorithm at blocks 750 and 760, allows the recommendation engine to produce parameter sets 
which evolve over time based on user behavior, 

Summarv of the Cited References 

U.S. Patent No. 6,029,195 to Herz 

Herz discloses a system for customized electronic identification of desirable objects. The 
desirable objects are described as including items such as news articles. The system automatically 
constructs both a "target profile" for each target object based on the frequency with which each 
word appears in an article relative to its overall frequency of use in all articles, as well as a "target 
profile interest summary" for each user, which describes the user's interest level in various types of 
target objects. The system then evaluates the target profiles against the user's target profile 
interest summaries to generate a user-customized rank order listing of target objects most likely to 
be of interest to each user so that the user can select from among these potentially relevant target 
objects, which were automatically selected by the system from the plethora of target objects that 
are profiled on the electronic media. User's target profile interest summaries can be used to 
organize the distribution of information in a large scale system consisting of many users 
interconnected by means of a communication network. Additionally, a cryptographically-based 
pseudonym proxy server is provided to ensure the privacy of a user's target profile interest 
summary, by giving the user control over the ability of third parties to access the summary and to 
identify or contact the user. 
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In contrast to the present invention, Herz does not disclose, teach, or suggest the utiUzation 
of a genetic algorithm as part of a recommendation system. Instead, Herz utilizes "quality 
attributes," that have the normative property that the higher (or in some cases lower) their value, 
the more interesting a user is expected to find them. (Herz, Col 19, lines 42-45.) A weight is 
associated with each attribute. (Herz, Col. 22, line 20.) The filtering system uses relevance 
feedback to refine its knowledge of user's interests: whenever the filtering system identifies a 
target object as potentially interesting to a user, the user provides feedback as to whether or not the 
target object really is of interest. (Herz, Col. 18, lines 16-18.) Applicant asserts that this system 
does not utilize a genetic algorithm. 

The Claims Distinguished 

Independent Claims 1,10, 20, and 25 

In regard to independent Claims 1,10, 20, and 25, applicant respectfiiUy submits that Herz 
fails to teach, disclose, or suggest each element of the amended claims. In particular, Herz fails to 
teach, suggest, or disclose the utilization of a genetic algorithm for adjusting sets of parameters. 

As described above, in one embodiment of the present invention the genetic algorithm is 
applied to a population of individual parameter sets (in one embodiment there are typically fifty or 
fewer parameter sets) in order to evolve the population in a usefiil direction. The population of 
parameter sets is evolved through a series of epochs. Each epoch includes key stages, including: 
fitness evaluation, death, breeding, mutation, and work. During the fitness evaluation, every 
individual parameter set in the seed population is evaluated using a specified fitness Amotion. 
After the fitness evaluation, a death fiinction prunes unsuitable individual parameter sets firom the 
population. Following the death fimction, a breeding session begins in which the probability of 
two individual parameter sets breeding is determined by specified factors. Once the mating 
partners are selected, their genes are combined to form a new individual parameter set. When a 
mutation stage is utilized, a specified probability is provided of whether a new parameter set may 
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undergo some form of mutation. At the end of the process, the population of parameter sets is 
available for work. At this point the system retrieves the population and uses the information 
encoded in the genomes of the individual parameter sets to generate recommendations for various 
media items. Following the work stage, a new epoch is entered and the process is repeated. Thus, 
through the use of the genetic algorithm, the recommendation engine is able to produce parameter 
sets which evolve over time. Applicant respectfully submits that Herz does not teach, disclose or 
suggest the utilization of a genetic algorithm as part of a recommendation system. Accordingly, 
applicant respectfully requests the withdrawal of the 35 U.S.C. § 102 rejection with regard to 
independent Claims 1,10, 20, and 25. 
Claims 12 and 28 

In regard to Claims 12 and 28, these claims depend from independent claims that recite the 
elements discussed above. Accordingly, appUcant asserts that they are allowable for the same 
reasons as discussed above. 

In addition, applicant points out that these claims recite "wherein the genetic algorithm 
executes unworthy parameter sets based on the performance of the recommendations." As 
described above, after a fitness evaluation of the parameter sets, a death function is utilized to 
prune unsuitable individual parameter sets from the population. As described in the specification 
on page 17 at line 1, an equation is given for the probability of an individual parameter set 
surviving from one epoch to the next. Applicant respectfully asserts that Herz does not teach, 
disclose, or suggest the execution of parameter sets based on the performance of the 
recommendations. Accordingly, applicant respectfully requests the withdrawal of the 35 U.S.C. 
§ 102 rejection with regard to Claims 12 and 28. 

Claims 13 and 29 

In regard to Claims 13 and 29, these claims depend from the independent claims that recite 
the elements discussed above. Accordingly, applicant asserts that they are allowable for the same 
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reasons as discussed above. In addition, Claims 13 and 29 require "wherein the genetic algorithm 
generates new parameter sets from the surviving population." 

As described above, following the death function, a breeding session begins. In one 
embodiment, the probability of two individuals breeding is determined by two factors. The first is 
the fitness values of the two individuals. The second factor is the individual bias that an 
individual parameter set holds for another parameter set. As described, in one embodiment 
individual parameter sets may be made to favor those other parameter sets that are similar to 
themselves, but not too similar, when mating. Once the mating partners are selected, their genes 
are combined to form a new individual. Applicant respectfully submits that Herz does not teach, 
disclose, or suggest the generation of new parameter sets from a surviving population. 
Accordingly, applicant respectfully requests the withdrawal of the 35 U.S.C. § 102 rejection with 
regard to Claims 13 and 29. 

Claims 34, 35, and 43 

Claims 34, 35 and 43 are dependent on independent claims that include the above recited 
limitations, and are therefore patentable for the same reasons discussed above. In addition, 
Claims 34, 35 and 43 require the "mating" of individual parameter sets. As described above, the 
mating of individual parameter sets depends on a number of factors. Applicant respectfully 
submits that Herz does not teach, disclose, or suggest the "mating" of individual parameter sets. 

Claims 36, 37 and 44 

Claims 36, 37 and 44 are dependent on independent claims that include the above recited 
limitations, and are therefore patentable for the same reasons discussed above. In addition, 
Claims 36, 37, and 44 require a survival stage, and an executing component, respectively, for 
pruning unsuitable parameter sets from the population. Applicant respectfully submits that Herz 
does not teach, disclose, or suggest a survival stage or an executing component. 
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Claims 38 and 39 

Claims 38 and 39 are dependent on independent claims that include the above recited 
limitations, and are therefore patentable for the same reasons discussed above. In addition, 
Claims 38 and 39 require that the initial population of parameter sets be created using either 
known values, or through a random process, respectively. As described above, in one 
embodiment a seed population of parameter sets is provided by the user. The seed population 
represents the user's initially known information about the parameter sets. If the user is 
completely ignorant about the parameters, then an initially random seed population of parameter 
sets can be utilized. Applicant respectfully submits that Herz does not teach, disclose, or suggest 
the creation of an initial population of parameter sets using either known values or a random 
process. 

Claims 40 and 45 

Claims 40 and 45 are dependent on independent claims that include the above recited 
limitations, and are therefore patentable for the same reasons discussed above. In addition, 
Claims 40 and 45 require a stage or component for mutating the parameter sets. As described 
above, in one embodiment, a newly created parameter set has some probability of undergoing 
some form of mutation. Mutations allow a population of parameter sets to explore new segments 
of a solution space. Without mutations, a population's characteristics may eventually freeze. 
However, the mutation rate should be chosen carefully. For floating point genes, in one example 
the mutation depth could be on the order of a few percent. In other words, the value of the floating 
point number changes by a few percent. Applicant respectfully submits that Herz does not teach, 
disclose, or suggest the utilization of a stage or component for mutating parameter sets. 

Claims 41 and 42 

Claims 41 and 42 are dependent on independent claims that include the above recited 
limitations, and are therefore patentable for the same reasons discussed above. In addition, 
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Claims 41 and 42 require that floating point, or binary genes, respectively, are utilized by the 
genetic algorithm. As described above, in one embodiment, each parameter set comprises a set of 
genes. Each gene represents a piece of information. In one embodiment, the only type of gene 
supported is a floating point gene. This gene is a convenient format for solutions that are 
expressed as a set of floating point numbers. In another embodiment, binary genes may be 
implemented. Applicant respectfully submits that Herz does not teach, disclose or suggest the 
utilization of floating point or binary genes. 



Based on the above-referenced amendments and arguments, applicant respectfully submits 
that all of the claims of the present application, Claims 1-3, 5, 10, 12-15, 20-23, 25, and 28-45 are 
allowable over the cited and applied references. Therefore, applicant respectfully requests 
withdrawal of all of the rejections of the claims and allowance of the present application. If any 
questions remain, applicant requests that the Examiner contact the undersigned at the telephone 
number listed below. 



I hereby certify that this con-espondence is being deposited with the U.S. Postal Service in a sealed envelope 
as first class mail with postage thereon fully prepaid and addressed to Mail Stop Amendment, Commissioner for 
Patents, P.O. Box 1450, Alexandria, VA 22313-1450, on the below date. ( ^ 



CONCLUSION 



Respectfully submitted, 

CHRISTENSBN O'CONNi 
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T/acy S. Powell 
Registration No. 53,479 
Direct Dial No. 206.695.1786 



Date: 
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